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ABSTRACT 



This study investigated Brazilian elementary teachers' 
decision making processes when evaluating and grading students' written work. 
Teachers analyzed the written work of four students who were unknown to them, 
then produced reports indicating: their perception of the author's grade 
level and reasons for believing it was that grade level; positive and 
negative aspects of the written piece; and the grade they would give for the 
work. Results indicated that the most visible characteristics of the texts 
(those most closely related to teaching contents* and correction) determined 
how consistently teachers rated the students ' grade levels and the grades 
that they gave to the texts. There were four evaluative patterns 
corresponding to: teachers producing reports that favored more formal and 
surface elements (orthography, punctuation, and paragraphs) and those related 
to such factors as theme development, cohesiveness, consistency, style, and 
genre; identification of reports that emphasized the formal aspects of the 
texts rather than the more subjective aspects; prevalence of informal or 
subjective aspects of the text over formal ones; and teachers* claims about 
needing to get to know the students and the contextual factors in order to be 
able to produce an adequate report. (Contains 13 references.) (SM) 
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This work is about the classroom assessment practices 
involved in teachers' decision-making processes when 
analyzing student's writings material (dissertations). Its 
objective was to analyze a specific teaching activity 
performed by teachers in everyday classroom, that is, to 
assess students' work, judge its worth and attribute marks 
(grades) . 

It is related to a broader applied research project 
developed by researchers from the Universidade Federal de Sao 
Carlos in partnership with teachers from the initial grades 
of an elementary school in a medium-size town in Sao Paulo 
State, Brazil. The project axis is the investigation of 
teachers' professional learning and development processes. 

The goals of the broader project are twofold; to build a 
knowledge base on teachers' learning and professional 
development processes and to define when, where and how the 
participants intervene in these processes with the objective 
of refining this knowledge. 

A constructive-collaborative approach (Cole & Knowles, 
1993) has been adopted, which presumes that the improvement 
in teaching quality to overcome school and student failure 
implies natural and voluntary participation of teachers in 
the discussion of alternative propositions and aims to 




BEST COPY AVAILABLE 



2 



accomplish such goals. This approach implies, among other 
assumptions : 

a) the concept of teachers' professional development 
considered as part of a continuum that seeks to establish 
connections between initial and continuing teacher education; 

b) the valuing of teachers' professional development 
processes, of contextual and organizational aspects, directed 
towards change and combining individual and collective 
dimensions of the pedagogical activity; 

c) the construction of teaching knowledge as a result of the 
dialectical relation between the individual and the 
collective; 

d) the inquiry-reflection principle (Knowles, Cole, & 
Presswood, 1994), which facilitates teachers' understanding 
of their pedagogical practice, considers the collaborative 
nature of roles impersonated by their peers, acknowledges the 
specificity of the pedagogical practice as requiring non- 
standardized solutions, admits the influence of teachers' 
conceptions in the understanding of classroom events and 
their teaching practice and enables the development of 
personal and professional autonomy, among other aspects; 

e) ... new forms of partnership research [...] based on 
fundamental assumptions on the importance of mutuality in 
purpose, interpretation, and reporting, and on the potency of 
multiple perspectives. It is also implicit in this model the 
understanding that each partner in the inquiry process 
contributes particular and important expertise , and that the 
relationship between the classroom teacher and the university 
researcher , for example, is multifaceted and not powerfully 
hierarchical (Cole & Knowles, 1993:478;; 

f) the need to establish a base knowledge that makes 
professional development possible; 

er|c 
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g) the consideration of processes of pedagogical knowledge 
content construction in different subject matters; 

h) the perception of the school as a learning organization; 

i) the construction of teachers' professional empowerment 
processes; 

j) the consideration of translations and transpositions of 
educational public policies related to schools and 
classrooms . 

1 . Theoretical Orientation 

In the broader project we have adopted - as theoretical 
orientation to understand the teachers pedagogical practices 
- the literature related to teacher thinking, to 'reflection 
as a conceptual orientation' (Valli, 1992) and to knowledge 
base itself (Shulman, 1986, 1987; Schoenfeld, 1997). 

Learning to teach is a complex process that involves, 
among other aspects, affective, cognitive, ethical, and 
performance-related factors (Cole & Knowles, 1993) . 

Studies on teachers' thinking, reflective teaching, 
knowledge base for teaching and tacit/practical theories 
etc., though based on different theoretical and 
methodological orientations, have pointed to the constructive 
dimension of professional knowledge, i.e., to the fact that 
this knowledge is developed during teaching, and to the 
personal dimension added to this constructive process. 

Studies have also indicated that teachers' knowledge, 
beliefs, and goals are fundamental elements when determining 
how they act in the classroom and why they act in such way. 
Learning to teach is a developmental process and teachers 
need time and resources to modify their practice. The 
modifications that teachers need to accomplish to be able to 
contemplate new social and political demands go beyond 
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learning new techniques and imply conceptual revisions of 
educational and instructional processes and of the practice 
itself . 

Based on these principles, part of the activities 
carried out along the broader project referred to the study 
of teachers' conceptions about a variety of aspects related 
to their teaching practice. The basic assumption was that 
teachers, upon describing, analyzing and inferring from 
classroom events, would create their own pedagogical 
principles through reflective processes. 

By considering the theme "evaluation" this work 
presupposes that changes are frequently necessary in 
classroom pedagogical practices for the evaluation process to 
be part of a successful learning process (Shepard, 2000) . 

Traditionally, when talking about evaluation, we 
primarily refer to results obtained by students as compared 
to the objectives set at different school levels. Zabala 
(1998) believes that evaluation has been considered as a 
sanctioning and qualifying tool, in which the subject of 
evaluation is the student and the student alone, and the 
object of evaluation is the learning accomplished according 
to certain minimal objectives set to all students (p. 195) . 

According to Perrenoud (1999) it would be more 
convenient to speak of "formative observation" instead of 
evaluation because the latter concept is associated with 
measures, ratings, school reports and the idea of 
transmissible and codifiable information that accounts for 
knowledge (p.l04). On the other hand, "formative observation" 
will allow teachers to direct and optimize ongoing learning 
without having to worry about rating, certifying and 
selecting (p.l04), because what matters is their coordinating 
function. What counts in this type of observation is the 
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theoretical frame that guides teachers and governs their 
interpretation of that that is observable (p.105). 

Perrenoud ( 1999) maintains that teachers, after teaching 
defined contents, usually carry out evaluations by means of 
procedures related to learning assessment and the qualitative 
or non-quali tative evaluation of students' performance. 
Later, these marks constitute profiles or averages that 
contribute to the decision about students' promotion or 
certification at the end of a study period, grade or cycle. 
In general, when carrying out this process teachers assess 
strictly individual performances from standard and closed 
questions. The "risk of arbitrariness" increases as they 
assess qualitatively different productions or more open 
questions (p.73). In the case of dissertations, the marks 
often reflect the evaluator' s tastes and caprices regarding 
students' competencies (p.73). 

Apparently, two variables seem to have a significant 
weight in assessment processes carried out by teachers: the 

student' s individual process and the perception of progress 
along the teaching-learning process (Mead (1992). From the 
conception that the mark/score is a visible feature with 
respect to the quality of students' productions, this author 
showed that this activity had special characteristics such 
as: it was randomic; it presented alternative systems; it 

demanded a set of information so as not to harm students' 
future learning nor arise feelings of uneasiness on the 
teacher's side. 

Other aspects, also considered in this work, were 
defined by Mead (1992) as constituting the bases adopted by 
teachers in assessment and grading processes. 

On the one hand there is the objective counting of 
correct and wrong answers. In this case teachers seem to take 
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into account internal inference sources when they count the 
mistakes or evaluate the cleanliness of some homework, for 
instance. These internal inference sources constitute a set 
of visible, obvious and concrete evidences that prevent 
strange factors from coloring the judgment and accompany the 
assessment process. 

On the other hand there is the qualitative count, which 
involves inferring from the student' s effort and 
understanding degree in relation to a given content. In this 
specific case, the student's past performance - such as 
his/her past school performance - or the time of the year the 
work assessed refers to seems to play a crucial role. Thus, 
if it occurs at the beginning of a given teaching-learning 
process, teachers seem to be more tolerant toward the type of 
student production. Contrarily, if students are in a more 
advanced phase, teachers are less tolerant toward mistakes. 
This qualitative count seems to derive from external 
inference sources, which may consist of more subjective and 
intuitive aspects that transcend existing evidences in the 
assessed production. In this case, for instance, the 
students' general behavior, personality traits and 
comprehension degree may be considered when carrying out a 
more global evaluation on students' performance. 

2 . Methodological Orientations 

2.1. Research question and issues under investigation 

Considering the nature of the broader project and the 
theoretical and methodological orientation it is supported by 
- which by itself implies a decision-making process -, during 
the project's first year of development strategies for 
research and intervention were constructed, which the group 
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named 'teaching and learning experiences' . They have been 
used throughout the years of this project's existence, and 
can be defined as: 

[...] structured situations of teaching and 
learning, planned by the researchers and 
schoolteachers, implemented by the 

schoolteachers, collectively discussed, 
originating from issues chosen by them, 
individually or by the group. These are 
experiences with a beginning, middle, and an 
end— and may be developed by small groups or 
by each teacher with his/her students. These 
experiences are born from practical 
difficulties related to different subject 
matters; challenges , which may arise from 
school daily activities; and public policies 
(Mizukami et al., 1998, p. 3). 

The present study is specifically about one of these 
teaching and learning experiences, which we have called 
" Assessment of the Teaching-Learning Processes ” . One of its 
primary goals is to obtain answers to the following research 
questions : 



What information on students' 
production is considered relevant by 
teachers of initial grades (marks) in 
the evaluation and grading process? 
What is its nature? Is the same 
information considered relevant by 
many teachers? 



2.2. Participants 

This teaching and learning experience involved 27 
professionals from the elementary school (first four grades) 
Escola Luiz Augusto de Oliveira - its principal, pedagogical 
coordinator and 25 teachers. All participants were female; 6 
out of the 25 schoolteachers taught first grade, 6 second 

grade, 6 third grade and 7 fourth grade. 
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2.3. Design and methodology 

This work focuses on investigating the teachers' responses 
to the task of assessing and attributing marks to the written 
production of four students. After careful analysis of one of 
the dissertations considered, each teacher handed in a 
written report indicating, with arguments, the grade attended 
by the author, the positive and negative aspects of his/her 
production and attributed a mark. 

The dissertations evaluated were produced by students 
from another school, because it was believed that it would be 
interesting to follow how the teachers being investigated 
would behave toward the production of unknown students. The 
dissertations subjected to the teachers' scrutiny were 
selected by a Portuguese Language teacher from a larger set 
of dissertations produced by students from different school 
grades. There were chosen dissertations that expressed 
different levels of thought and reflection and that showed 
several patterns of orthography, punctuation, use of capital 
letters, structuring and consistency. 

Considering the nature of the research problem, this 
work may be characterized as having a qualitative approach 
whose data (teachers' written answers to a set of questions) 
were given a descriptive-analytical treatment. 
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3. Assessment patterns shown by the teachers and intervening 
variables 

The general analysis of the data obtained indicates that 
the most visible characteristics of the texts, those more 
closely related to teaching contents and correction, seem at 
first to determine a greater consistency of the evaluative 



9 



9 



tendencies verified among the teachers as to the indication 
of the school grades the text authors were thought to be in 
as well as the marks corresponding to the quality of the 
text. 

In order to illustrate the point, we can cite the 
results referring to Text D as compared to those of Text C. 
In the latter case, the visible evidences related to internal 
inference sources (adequate theme development; text with 
beginning, middle and end; adopted logic; consistency; 
separation of ideas into paragraphs; correct punctuation and 
orthography) were more consistently pointed by the teachers. 
For example, with respect to the indication of the grade the 
authors were in, they were almost equivalently divided as 
being in the grade (13) or 4^^ grade (14) . In the case the 
student was in the grade, 53.8% (7) of the teachers 

attributed A to him/her, 38.4% (5) B e one teacher C. In case 

he/she was in the 4^^ grade, 42.9% (6) of the teachers would 

attribute A and the same percentage B, and 14.3% (2), C. 

Contrarily, in the case of Text D the attributes 

considered visible, but considered as mistakes, were not so 
obvious for the teachers to present a more precise evaluative 
report. In this specific case the text was ascribed as being 
from a I®^ grade student to 29.6% (8) of the teachers, 2"^ 
grade to 25.9% (7), 3^^ grade to 22.2% (6), 4^’" grade to 7.4% 

(2), 2"^ or 3*^^ grades to 11.1% (3) and 2"^, 3^^^ or 4^^ grades 

to 3.3% (1) . The same type of divergence was observed in 

^^i^ge^ *t^o% 

grade attributed C to the text. One hundred percent (7) of 
the teachers that indicated that the author was a 2"*^ grade 
student gave him C. The teachers that believed he was a 3'^'^ 
grade student (66.7% or 4) also gave him C. Differences among 
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the teachers' personal conceptions about the mastery of an 
ability at a defined level are certainly the source of such a 
variety of judgments. 

A set of variables stood out from the answers presented 
by the teachers and was considered as directly corresponding 
to internal and external interference sources related to the 
material and its authors. However, a third source of 
variables related to the evaluator him/herself was observed. 

In the case of Internal Inference sources, the 
identified variables corresponded to superficiality elements 
such as the presence of orthographic mistakes, wrong use of 
punctuation and capital letters by the authors of the texts. 
The presence of textuality elements was apparently another 
element considered by the teachers, despite being more 
subjective than the previous one. It included structuring the 
text into paragraphs and the development of the theme. 
Somehow it is possible to say that these two aspects relate 
directly to the teaching contents usually worked by the 
teachers. 

In the case of external inference sources it was 
observed that the personal characteristics related to the 
authors of the text, such as effort, the grade they were in 
and time of the year when the text was produced, were 
mentioned as relevant aspects. 

The evaluator' s positioning as regards the material 
analyzed may have constituted another set of factors that 
influenced the evaluations. On the one hand, some reports 
evidenced a high degree of security as to their adequacy 
suggesting that the evaluators "were positively sure" about 
their judgments. In these cases the grade they taught also 
seems to be an important aspect to be considered. On the 

other hand, some teachers often reported "doubts" and 
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"dilemmas" as to the decisions they made. In this case, some 
teachers clearly indicated the establishment of hypotheses 
about characteristics related to the students with respect to 
the results of their evaluations. 

Combinations of the variables relative to the material 
itself, to the authors of the texts and to the teachers, 
subjects of this investigation, enabled the outlining of the 
following evaluative patterns. 

The classification below took into account the 

predominating pattern of each one of the teachers when 

evaluating the four texts, which prevents affirmations as to 
the existence of "pure" patterns among them. 

The first pattern corresponds to the teachers imparting 
reports that favored more formal and surface elements 
(orthography, punctuation, paragraphs) and those related 
to the textuality (e.g., theme development, cohesion, 

consistency, style, genre) . Seldom did they mention any 
characteristics of the students. The teachers showing this 
pattern did not indicate they had doubts about the 
analyses they made. In this category T5, T8, Til, T16, 

T18, T20, T25, T30 and the school principal were included. 
An example of this pattern is one of T5' s reports, about 
Text D. 

I believe that the student is in the grade. 

He/ she writes consistently, without using correct 
punctuation. He/she wrote the whole story in just 
two paragraphs and made major orthographic 
mistakes . The mark would be C. I considered as 
positive the fact that the text made sense (the 
student was able to express him/herself) . I 
considered as negative points the paragraph 
organization , punctuation , orthography, 

consistency with the title (the student only 
spoke about the games and did not speak about the 
birthday itself). T5 , Text D. 
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• The second pattern corresponded to the identification of 

reports that also emphasized the formal aspects of the 

texts as compared to the more subjective ones. However, in 

this case indication of the students' characteristics was 

observed. The analyses included the teachers' 

manifestation of some sort of doubt or uncertainty. 

Nevertheless, the expressions indicative of security as to 

the performed evaluation usually prevailed over those that 

suggested some degree of doubt or uncertainty. This 

pattern was observed in the analyses made by T3, T14, T22, 

T28, T31. For instance, see one of T31's reports below. 

He/she could be a child beginning to read and 
write. Second grade, maybe 3^^^ grade. He/ she makes a 
lot of orthography mistakes . In spite of the fact 
that the text displayed some logic, structure , 
punctuation, it was off the theme. I would mark it 
C because in the beginning of the text the ideas 
are not explained well . It presents some 
definition as regards the paragraphs , but he/she 
did not develop the plot well. Positive point - 
the idea of the text. Negative point - the 
orthography mistakes . T31, Text B. 

• The third pattern type identified in this work is strongly 

characterized by the prevalence of informal or subjective 

aspects of the text over the formal ones. The reports 

usually enhance the characteristics related to the authors 

of the texts, such as their efforts to do it right. In 

some cases the teachers' dilemmas and doubts appear 

especially in relation to the grades the students are in. 

The analyses of T2, T4, T9, T12, T27 and the pedagogical 

coordinator are included in this category. The following 

examples illustrate this pattern. 

Third grade because of the organization and 
elaboration of the text ideas. The text also 
presents some important characteristics such as 
paragraphing, punctuation and meaning. The mark 
would be B, because the student still has a long 
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way that will allow for greater enriching. 
Positive points - the student's frankness ^ his 
effort in the elaboration of a very informative 
text on what he/ she considers to be a "A Sports 
Sunday". Negative point - I have nothing to add 
because this is a continuous process. T2 , Text A. 

He/ she could be a 2^^^ 3^^ or even 4^^ grade 

student. The text is consistent, but [presents] 
some orthography mistakes that students from all 
grades can make. If the child is in the 2^^ 
grade, I would give him/her a C, because the 
student only used two paragraphs , but it is 
possible to understand it. While if the student 
is in the 3^^ or 4^^ grade the mark would be D, 
because the student needs a lot of improving. 
Positive points - it is a text the reader is able 
to understand, he/she employed parentheses for 
the explanation. Negative points - it does not 
have much to do with the title, he/she only 
talked about the games they played. T9, Text D. 

• The fourth pattern is characterized by the teachers' 
claims related to the need of getting to know the 

students, authors of the texts, and the context elements 
so as to be able to produce an adequate report. In these 
cases, the analyses occasionally contain hypotheses. The 
reports of Tl, TIO, T13, T19, T24 and T32 are included in 
this pattern. One of T19's reports illustrates this 

pattern. 

4^^ grade, because I have students at different 
levels that write very well and others that are 
still learning how to read and write. Without 

knowing the student it is hard to assess his/her 
work, because it depends on the level he/she was 
at the beginning of the year, whether he/ she 
progressed or not. However, thinking of a student 
I have I would mark: Great! T19, Text A. 

It is important to mark that some teachers did not 
indicate negative aspects in the texts they analyzed. The 

explanation for this attitude was the present educational 
system, in the form of cycles, the dynamics related to 
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assessment in the continuous progression system and the 
theoretical references officially adopted by the school as 
the base to understand the teaching-learning process, in this 
case constructivism. 

The apparent emphasis on learning as a continuous 
process may at first be adequate, since the student's 
learning pace is respected. On the other hand, it may 
evidence the difficulties faced by the teachers when defining 
their role in this process. T2's report on Text D exemplifies 
this situation: ... I do not consider [negative points] 
because the student has in his/her favor the time that 
present education allows. Or T32's, with respect to the same 
text: ... negative points - I cannot consider [that] because 
it is the continuous education phase and we have up to the 4^^ 
grade to try to correct these mistakes . T28 affirms about 
Text A: ... negative points - it is impossible to speak about 
negative points when the whole system adopts the continuous 
progression and says that students will learn how to read and 
write along Cycles I and II, otherwise along their lives. 

4. Some final considerations 

By trying to understand what the teachers would engage 
in to correct and attribute marks to the students' 
productions in situations outside their context - when 
lacking the knowledge and experiences constructed from 
personal interactions - it was possible to evidence that the 
aspects related to teaching contents constituted the 
assessment base of the students' performance. 

When the teachers addressed teaching contents, it seems 
that they had in mind the topics to be taught at each level, 
even if it did not coincide entirely with what they thought. 
Different degrees of demand were evidenced among the 
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teachers, and they set out from these degrees to indicate the 
grade to which they thought the students whose work they had 
analyzed belonged. These degrees may be functionally related 
to variables identified as intervening in the assessment 
processes by means of this study. On the one hand, the data 
obtained confirmed that the assessment practice is permeated 
by ambiguity (Perrenoud, 1999) . On the other hand, they draw 
the attention to the fact that teaching programs usually 
prescribe what should be taught, though not what should be 
assessed in the different phases of the learning process. 

As regards the rigor in the correction of the texts of 
unknown children it is possible to verify a variety of 
degrees supported by different arguments: the highest rigor 
seems to be based, for most teachers, on text surface 
questions, regardless of the grade the student was in (which 
does make a difference) , and the lowest rigor is repeatedly 
justified by the existence of the cycles and, thus, a lot of 
time for the children to mature (which minimizes their 
responsibility over promoting their students' learning) . 
Further investigation in how these situations constitute may 
bring new elements to understand the "arbitrariness risk" 
mentioned by Perrenoud (1999) . 

Considering the attribution of marks on the part of the 
teachers being investigated, there is evidence that the marks 
were attributed based on a set of rationalizations, though 
the global assessment was the most valued parameter. However, 
the mark attribution should be regarded as a specific moment 
in the assessment process (Zabala, 1998; Perrenoud, 1999). 

The proposition of non-invasive activities (Knowles, 
Cole & Presswood, 1994) - such as the assessment of unknown 
students' productions - however close to the teaching 

imparted by the professionals seemed to be an important 
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formative strategy (Schoenfeld, 1997; McDiarmid, 1995; 
Argyris & Schon, 1996) . These activities offered some of the 
benefits observed in more contextual formative situations, 
since they also confronted deeper beliefs and escaped the 
censorship of the established discourse (Clandinin & 
Connelly, 2000) . 
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